To solve the problem that sparse coding in text image restoration has the shortcomings of limited expression of dictionary atoms and high computation complexity, a novel text image restoration algorithm was proposed based on sparse coding and ridge regression. Firstly, patches were used to train the dictionary for sparse representation at training stage and the sampled image were clustered based on the Euclidean distances between the sampled image patches and the dictionary atoms. Then, the ridge regressors between low-quality text image patches and clear text image patches were constructed in local manifold space to achieve the local multi-linear expansion of dictionary atoms and fast calculation. At last, the clear text image patches were directly calculated at testing stage by searching for the most similar dictionary atoms with low-quality text image patches without calculating the sparse coding of low-quality text image patches. The experimental results show that compared with the existing sparse coding algorithm, the proposed algorithm has improved Peak Signal-to-Noise Ratio (PSNR) by 0.3 to 1.1 dB and reduced computing time at one or two orders of magnitude. Therefore, this method provides a good and fast solution for text image restoration.